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Aggregation 


Introduction 

A recurring issue in quantitative risk assessment and quantitative risk characterization is the 
aggregation (and disaggregation) of separate but related causes and effects of risk. Questions 
about the aggregation of causes or agents differ somewhat from questions about the aggrega¬ 
tion effects or end points, but the similarities are great enough for us to treat them together in 
this chapter. For example, people may be exposed to mixtures of compounds from a single 
stack, and each compound may be associated with an increase in the degree or probability of 
occurrence of one or more toxic end points; the situation can be further complicated by 
questions about synergy. In contrast, dose-response data are often available only on single end 
points in response to doses of single agents. How should we characterize and estimate the 
potential aggregate toxicity posed by exposure to a mixture of toxic agents? 

The aggregation problem is simplified when all end points of concern are believed to have 
dose-response thresholds or no-adverse-effect levels. Under this restriction, "acceptable," 
"allowable," or "reference" doses are typically calculated by dividing empirically determined 
threshold estimates (such as no-observed-adverse-effect levels, NOAELs) by appropriate safety 
or uncertainty factors (Dourson and Stara, 1983; Layton et al., 1987; Barnes and Dourson, 
1988; Lu, 1988; Shoaf, 1991). The risk-management goal for mixed exposures is generally to 
avoid exposures that exceed any of the relevant thresholds, while taking into account the 
possible joint effects of multiple agents. One strategy that has been implemented in environ¬ 
mental and occupational settings is to ensure that the sum of all the ratios of incurred dose to 
acceptable dose relevant to a given end point total less than 1 (NRC, 1972a, 1989; OSHA, 
1983; ACGIH, 1977, 1988; EPA, 1987a, 1988g; Calabrese, 1991; Pierson et al., 1991). That 
approach is based on an assumption that doses of different agents can be treated as roughly 
additive with regard to inducing the end point; this assumption is reasonably consistent with 
much of the experimental evidence on the joint actions of chemicals in mixtures. 

Among the key problems associated with the general strategy is that the procedures currently 
used for defining acceptable exposures to systemic toxicants are rather crude. Proposals to 
incorporate more quantitative treatment of data and to focus on risk prediction without 
reference to thresholds (e.g., Crump, 1984; Dourson et al., 1985; Dourson, 1986) have not 
been widely adopted. The additivity assumption for systemic toxicants further complicates the 
crude approaches taken to identifying safe intakes of the components of a complex mixture. 
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As an Environmental Protection Agency (EPA) technical support document (EPA, 1988g) 
comments, this use of the additivity assumption implies that, 


as the acceptable level is approached or exceeded, the level of concern increases linearly . . . and in the same manner 
for all mixtures [which is incorrect, because the estimates used to derive such recommended acceptable levels] do not 
have accuracy or precision, and are not based on the same severity of toxic effect. Moreover, slopes of dose- 
response curves in excess of [such levels] in theory are expected to differ widely. The determinations of accuracy, 
precision or slope are exceedingly difficult because of the general lack of toxicity data. 


Despite its drawbacks, the crude additivity approach to the problem of aggregation of potential 
threshold effects has had relatively straightforward and uncontroversial regulatory applications. 

Much more debate has focused on quantitative risk-assessment methods for end points 
assumed not to have threshold dose-response relationships, such as cancer. Particularly with 
regard to environmental exposures to multiple chemicals, risk-management decisions (e.g., 
clean up criteria) tend to be driven by the estimated low-dose risk associated with exposure to 
materials that lead to assumed nonthreshold end points. This chapter focuses on aggregation of 
different risks and different types of risk attributable to integrated, multiroute exposure to 
multiple chemicals that are assumed to have nonthreshold effects. 


Exposure Routes 

Any comprehensive assessment of health risk associated with environmental exposure to any 
particular compound must consider all possible routes by which people might be exposed to 
that compound, even if expected applications in risk management are limited to some particu¬ 
lar medium, such as air, or particular source generator or category, such as a coke-oven 
facility. That is because compounds present in one environmental medium might be trans¬ 
ferred to another at any time before exposure. The major routes of exposure are inhalation, 
ingestion, and dermal absorption. In the context of environmental exposures, inhalation 
per tains to uptake of compounds present in respired air during rest or activity both indoors and 
outdoors; ingestion refers to gastrointestinal absorption of compounds that are intentionally or 
unintentionally present in any ingested material, including water, liquid foods, mother's milk, 
solid foods (including crops and game), and soil; and dermal absorption refers to percutaneous 
uptake of compounds deposited on skin, including those present in water during showering, 
bathing, or recreational swimming. Assessments of exposure to a substance from a given 
source must account for all potentially important routes by which the substance might come 
into contact with people (or environmental biota, if an ecological impact assessment is being 
undertaken). For example, mercury emitted into air from an industrial smoke stack might be 
inhaled by nearby residents, but might pose an even greater health risk by the ingestion of 
bioconcentrated mercury in fish that are caught locally after mercury from the stack plume has 
been deposited onto lake water. 

EPA has given the issue of integrated multiroute exposure considerable attention in the 
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context of risk-assessment guidance for Superfund-related regulatory compliance (EPA, 
1989a). For example, EPA suggested that assessment of the environmental fate and transport 
of compounds in ambient air address a range of issues as diverse as volatilization and occur¬ 
rence in wild game (EPA, 1988h, 1989a,c,d,e). Additional information on multimedia 
transport and multiroute exposure assessments is available (Neely, 1980; Neely and Blau, 
1985; Cohen, 1986; McKone and Layton, 1986; Allen et al., 1989; Cohen et al., 1990; 
McKone and Daniels, 1991; McKone, 1991, 1992). 


Risk-Inducing Agents 

Quantitative environmental risk assessment is often needed for exposure to multiple toxic 
agents, for example, in the context of hazardous-waste, drinking-water, and air-pollution 
control. The 1990 Amendments to the Clean Air Act in particular, list 189 airborne pollutants 
of immediate regulatory concern that can be emitted singly or in combination from a variety of 
specified emission-source categories. 

Over the last 2 decades, environmental remediation involving complex chemical mixtures 
has required general reviews of issues and cases of potential toxicity associated with concurrent 
exposure to multiple chemical agents (e.g., NRC, 1972a, 1980a,b, 1988a, 1989; EPA, 1988i; 
Goldstein et al., 1990; Calabrese, 1991). The earlier reviews supported the concept that 
toxicity predicted by dose additivity or concentration additivity was reasonably consistent with 
data on the joint action of acute toxicants (NRC, 1972a, 1980a,b; ACGIH, 1977; EPA, 

1987a). Although some cases of supra-additivity for acute toxicants are known, such as the 
synergistic interaction of organophosphate pesticide combinations in which one compound 
inhibits the detoxification of another compound, additivity has nevertheless been viewed as a 
reasonable expectation at the low doses at which detoxification enzymes are not expected to be 
saturated (NRC, 1988b; Calabrese, 1991). 

The EPA Database on Toxic Interactions, as of 1988, covered 331 studies involving roughly 
600 chemicals (EPA, 1988g). Most of the studies focused on the effects of two-compound 
mixtures on acute lethality; fewer than 10% examined chronic or lifetime toxicity. Less than 
3% of all the studies reported clear evidence of a synergistic interaction—i.e., a "response to a 
mixture of toxic chemicals that is greater than that suggested by the component toxicities" 
(EPA, 1988g). However, EPA also concluded that in only one of 32 studies chosen as a 10% 
random sample of the 331 studies was the design and use of statistics "appropriate with the 
conclusion justified" (EPA, 1988g). As a consequence, EPA has asserted that 

"given the quality and quantity of the available data on chemical interactions, few generalizations can be made 
concerning the likelihood, nature or magnitude of interactions. Most interactions that have been quantified are within 
a factor of 10 of the expected activity based on the assumption of dose addition’' (EPA, 1988g). 

Results of the few detailed comparative studies in which Salmonella-mutation assays were 
applied to complex mixtures (kerosene-combustion particles, coal-hydrogenation material, and 
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heterocyclic amines from cooked food) are also generally consistent with approximate 
additivity of mutagenic potencies of constituents within complex mutagenic mixtures (Thilly et 
al. f 1983; Felton et al., 1984; Schoeny et al., 1986). 

Epidemiological evidence concerning the synergistic potential of human carcinogens (usually 
involving long-term cigarette-smoking) has been extensively reviewed (Saracci, 1977; 
Steenland and Thun, 1986; EPA, 1988g; NRC, 1988a,b; Kaldor and L’Abbe, 1990; Pers- 
hagen, 1990; Calabrese, 1991). Although no single mathematical expression is likely to give 
an accurate representation of joint effects, especially given the heterogeneity of human 
responses, the discussion here has often focused on whether responses are more clearly 
additive or multiplicative. The best-studied interactions (such as in joint exposure to tobacco 
and radon or tobacco and asbestos) suggest that a strictly additive model within the dose ranges 
may underestimate the true joint effects by a factor of 3-10. Results of epidemiological 
studies of joint exposure to radon progeny and cigarette smoke, for example, have been 
interpreted as showing an additive or possibly multiplicative interaction of the two agents with 
respect to the number of cancers induced and a synergistic decrease in latency period for tumor 
induction (NCRP, 1984; NRC, 1988a). The NRC (1988b) BEIR IV committee concluded that 
results of epidemiological studies of smoking and nonsmoking uranium miners exposed to 
radon gas, particularly the large study by Whittemore and McMillan (1983), were consistent 
with a multiplicative effect of the combined agents. 

The effects of asbestos exposure among workers who have a history of cigarette-smoking 
have been described (NRC, 1988a) as "one of the most current and well-recognized examples 
[based on epidemiological data] of how two distinct agents administered together can produce 
an increased incidence of [lung] cancer that is greater than that predicted from the administra¬ 
tion of either agent alone [and that] is considered multiplicative by most investigators who 
have studied the problem." A study not cited by NRC of more than 1,600 British asbestos 
workers suggests an additive, rather than multiplicative, increase in relative risk after joint 
tobacco and asbestos exposure (Berry et al., 1985). Other investigators have also concluded 
that the overall evidence of multiplicative interaction of these agents is questionable (Saracci, 
1977; Steenland and Thun, 1986). 

Epidemiological detection of possible multiplicative action among human carcinogens is not 
surprising, given the large amount of experimental data on the action of cancer promoters in 
animals, including clear examples of supra-additive interaction (EPA, 1988g; Calabrese, 1991; 
Krewski and Thomas, 1992). Highly nonlinear, supra-additive synergistic interaction of some 
types of nongenotoxic cancer promoters with genotoxic agents is predicted by "bio- 
mechanistic" multistage models of carcinogenesis. In those models, increased cell replication 
can play a pivotal role either by directly increasing the rates of production of premalignant or 
malignant lesions, by amplifying the incidence of malignant lesions through stimulated growth 
of spontaneously occurring premalignant lesions, or both (Armitage and Doll, 1957; Mool- 
gavkarand Knudson, 1981; Moolgavkar, 1983; Bogen, 1989; Cohen and Ellwein, 1990a,b; 
1991; Ames and Gold, 1990a,b; Preston-Martin et al., 1990). From that mechanistic 
perspective, several nongenotoxic compounds are now thought to be capable of promoting 
carcinogenesis, both spontaneous and experimentally chemically induced, solely by increasing 
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target-cell replication, a phenomenon that might have a threshold-like dose-response relation 
(Weisburger and Williams, 1983; Weisburger, 1988; Butterworth, 1989, 1990; Bogen, 1990b; 
IARC, 1991; Flamm and Lehman-McKeeman, 1991). EPA is considering formal recognition 
of such threshold carcinogens from the mechanistic perspective (e.g., EPA 1988g, 1991d), 
although these cases remain awkward to accommodate within EPA's currently-used 1986 
general scheme for classifying potential chemical carcinogenicity (EPA, 1987a). 

In general, both biological and statistical considerations make it difficult to rule out a 
nonthreshold mutation-related component of chemically induced carcinogenesis, and this effect 
might be dominant at low environmental exposures (Portier, 1987; Portier and Edler, 1990; 
Kopp-Schneider and Portier, 1991; Weinstein, 1991). For example, an increase in target-cell 
replication induced by some nongenotoxic chemicals might have a low-dose, linear, non¬ 
threshold dose-response relation. Alternatively, a broad distribution of thresholds within a 
highly heterogeneous human population might give rise to practical quasilinearity or super¬ 
linearity for low-dose promotional effects. Therefore, low-dose linearity has been recom¬ 
mended as a reasonable default assumption, even for agents known to increase cancer risk 
through nongenotoxic promotional mechanisms, in the absence of data establishing a pertinent, 
clearly defined, generally applicable threshold dose-response relation (Lutz, 1990; Perera, 
1991). Under this default assumption, the mechanistic type of cancer-risk model and the 
classical multistage cancer-risk model both predict that small amounts of increased risk will be 
approximately linearly proportional to the risk associated with small combined doses of 
genotoxic or nongenotoxic carcinogens, or both, and that their joint action will be approxi¬ 
mately additive (Gibb and Chen, 1986; NRC, 1988a; Brown and Chu, 1989; Krewski et al., 
1989; Kodell et al., 1991b). 

The general assumption of low-dose linearity for a presumed nonthreshold quantal end point 
(i.e., an end point observed only as present or absent), such as cancer occurrence before age 
70, is equivalent to assuming P = p + qD, where P is the risk of such occurrence after a 
lifetime exposure at dose rate D, p is the background cancer risk by age 70, and q is the 
potency (increased risk per unit dose) for small values of D. Of interest is the aggregate 
increased probability P of cancer occurrence due to exposure to a low-dose environmental 
mixture of nonthreshold toxic agents. If the linear model is assumed for each of two such 
agents, and if an additional independent-action assumption is made that the agents act through 
statistically independent events to increase risk R, it follows that P * q,D, + q^ for very 
small Dj and D 2 (NRC, 1980b, 1988b; Berenbaum, 1989). A more general sum of potency- 
dose products has been used by EPA for approximating P in cases of exposure to a mixture of 
carcinogens (EPA, 1987a, 1988g). Appendix 1-1 shows that the same general assumptions 
imply that a similar sum-of-products relation may be used to approximate the risk associated 
with mixtures of agents, each having one or more different end-point-specific effective dose 
rates. Multiple nonthreshold end points can be of interest in quantitative risk assessment, as 
discussed in more detail below. 
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Types of Nonthreshold Risk 


Quantitative risk assessment can involve multiple toxic end points, as well as multiple toxic 
agents. In particular, toxic end points other than cancer might at some point also be assumed 
to have nonthreshold dose-response relations for public-health regulatory purposes. Further¬ 
more, cancer is not a single disease, but a variety of neoplastic disorders with different 
characteristics that occur in different tissues of animals and humans at different times in the 
life history. Aggregate human cancer risk is often estimated from animal bioassay data that 
indicate statistically significant increases in dose-related risk of more than a single tumor type 
(e.g., cancer of the lung and cancer of the kidney). Similarly, genetic, reproductive, and 
developmental risks can arise in multiple forms that are measured separately in toxicity assays 
(e.g., reduced fertility and incomplete ossification of some bone). The issues of aggregating 
risk of both multiple end points and multiple types of a given end point are discussed below. 
Both these aggregation problems can be addressed simultaneously by using Expression 6 in 
Appendix 1-1, if independent actions and effects are assumed. 


Cancer 

The issue of how to use bioassay data that indicate dose-related effects for multiple tumor 
types is addressed by the EPA (1987a) cancer-risk guidelines as follows: 

To obtain a total estimate of carcinogenic risk, animals with one or more [histologically distinct] tumor sites or 
types showing significantly elevated. . . incidence should be pooled and used for [risk] extrapolation. The pooled 
estimates will generally be used in preference to risk estimates based on single types or sites. 

If different tumor types observed to have increased incidences are known to occur in a statisti¬ 
cally independent fashion within and among the bioassay animals tested, this EPA-recom- 
mended procedure leads to inconsistently biased estimates of aggregate potency or risk be¬ 
cause, under the independence assumption, the pooled tumor-incidence data may randomly 
exclude relevant information (Bogen, 1990a). For potency estimates based on classical 
multistage models, that statistical problem is avoided if aggregate potency is estimated as the 
sum of tumor-type-specific potencies (Bogen, 1990a). If the latter approach is used, then the 
aggregate increased risk P of incurring one or more tumor types at a very low dose can be 
estimated from Expression 7 in Appendix 1-1 (for one carcinogen). The type-specific poten¬ 
cies are uncertain quantities (one reason is that they are generally estimated from bioassay 
data), so appropriate procedures must be used for summation. 

This alternative (Expression 7 in Appendix 1-1) to EPA's procedure for estimating aggregate 
cancer potency depends on the validity of the assumption that different tumor types occur 
independently within individual bioassay animals. If substantial interanimal heterogeneity 
exists in susceptibility to cancer, or if tumor types are positively correlated, the occurrence of 
multiple tumor types would be expected to cluster in the more susceptible individuals. 
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Although some significant tumor-type associations have been identified in some species, they 
have tended to involve a relatively small number of tumor types (see Appendix 1-2). 

Appendix 1-2 summarizes an investigation of independence in interanimal tumor-type 
occurrence in a subset of the National Toxicology Program (NTP) 2-year cancer-bioassay data, 
which has been used by EPA as the basis for quantifying the potency of most chemical 
carcinogens. Separate analyses were conducted for four sex-species combinations (male and 
female mice, male and female rats) by using control-animal data from 61 rat studies and 62 
mouse studies and treated-animal data from a subset of studies in which there were significant 
increases in multiple tumor types. Correlations in the occurrence of pairs of tumor types in 
individual animals were evaluated. Little evidence was found of tumor-type correlation for 
most of the tumor-type pairs in control and treated mice and rats. Some tumor-type pairs were 
statistically significantly (and generally negatively) correlated, but in no case was the correla¬ 
tion large. These findings indicate that a general assumption of statistical independence of 
tumor-type occurrences within animals is not likely to introduce substantial error in assessing 
carcinogenic potency from NTP rodent-bioassay data. 


Other Nonthreshold End Points 

Two major categories of possible nonthreshold toxicity other than cancer that may often be 
relevant in quantitative risk assessment are genetic mutation (which might be caused by 
material that reaches and damages gonadal DNA) and developmental and reproductive toxicity 
(such as developmental neurotoxicity of lead). In general, however, if both dose-response 
linearity at low doses and independent dose induction of these effects are assumed, then they 
may also be incorporated with cancer into the general additive strategy already discussed. The 
extent to which those assumptions might apply to genetic toxicity and reproductive and 
developmental toxicity is considered below. 


Genetic Effects 

Mutagenic agents can cause detrimental inherited effects with an important genetic compo¬ 
nent, such as clinically autosomal dominant and recessive mutations, X-linked mutations, 
congenital birth defects, chromosomal anomalies, and multifactorial disorders of complex 
origin. Inherited genetic effects other than complex multifactorial effects have been found to 
occur spontaneously in roughly 2% of all livebom people, appearing either at birth or thereaf¬ 
ter; about 40-80% often involve chromosomal anomalies or dominant or X-linked mutations 
( M CADXMs“) (Mohrenweiser, 1991). In addition, more than 25% of all spontaneous 
abortions are thought to be due to genetic defects, the majority involving CADXMs (Mohren¬ 
weiser, 1991). Rates of those genetic effects are known to be increased in animals by 
exposure to environmental agents, such as ionizing radiation (which also causes cancer); 
furthermore, the risks of both genetic and cancer end points associated with low doses of 
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ionizing radiation are currently modeled as being increased above background in linear 
proportion to dose (NRC, 1972b, 1980c, 1990b; NCRP, 1989; Favor, 1989; Sobels, 1989; 
Vogel, 1992). Exposure of experimental animals to mutagenic chemicals can also cause some 
of these genetic effects, although specific characteristics of chemically induced genetic damage 
appear to differ in some ways from those induced by irradiation, e.g., in the fraction of 
dominant versus recessive specific-locus effects (Ehling and Neuhauser, 1979; Lyon, 1985; 
Favor, 1989; Rhomberg et al., 1990). 

Experimental data are not all consistent with a linear nonthreshold dose-response relation for 
genetic end points induced by either chemicals or ionizing radiation. (ICPEMC, 1983a; 

Sobels, 1989). Chemical mutagenesis, in particular, involves many potentially nonlinear and 
threshold processes, such as transport of reactants, metabolic activation and deactivation, DNA 
repair, and chemically induced functional change and lethality (ICPEMC, 1983a). However, 
it is diffic ult (if not impossible) to show experimentally that a complex, inherently statistical 
biological response does not differ from background (ICPEMC, 1983a). In light of such 
complexities, several National Research Council committees (NRC, 1975, 1977, 1983b) have 
concluded that the linear nonthreshold dose-response assumption used for ionizing radiation is 
also a reasonable default hypothesis for mutagenic chemicals. That conclusion reflects the fact 
that "if an effect can be caused by a single hit, a single molecule, or a single unit of exposure, 
then the effect in question cannot have a threshold in the dose-response relationship, no matter 
how unlikely it is that the single hit or event will produce the effect." It has been similarly 
concluded that a linear nonthreshold dose-response relation is a reasonable default assumption 
for chemical mutagens (Ehling and Neuhauser, 1979; ICPEMC, 1983a,b; Lyon, 1985; Ehling, 
1988; Favor, 1989; Sobels, 1989; Rhomberg et al., 1990). 

Such support of a default assumption of nonthreshold linearity in induced genetic risk has 
highlighted the uncertainty that exists in quantitative assessment of the total genetic risk to 
humans associated with exposure to ionizing radiation or genotoxic chemicals. That uncer¬ 
tainty, due particularly to problems in estimating possible increases in rates of human genetic 
disease, has led some to conclude that realistic assessment of total genetic risk associated with 
environmental exposure will not soon be possible (NRC, 1990b; Mohrenweiser, 1991; Vogel, 
1992). The degree of uncertainty varies greatly among different end points, but dose-response 
data for mutations in mice, supplemented by corresponding estimates of human spontaneous 
incidence rates, appear to provide a basis for reasonable quantitative risk assessment for some 
genetically simple and straightforward end points, such as those involving CADXMs (NRC, 
1990b; Mohrenweiser, 1991; Vogel, 1992). 

In 1986, EPA adopted guidelines for mutagenicity risk assessment that do not specifically 
endorse a linear nonthreshold default assumption. Rather, they state that EPA "will strive to 
use the most appropriate extrapolation models for risk analysis" and "will consider all relevant 
models for gene and chromosomal mutations in performing low-dose extrapolations and will 
chose the most appropriate model" (EPA, 1987a). The 1986 guidelines committed EPA to 
"assess risks associated with all genetic end points" to the greatest extent possible when data 
are av ailab le, with risk to be "expressed in terms of the estimated increase of genetic disease 
per generation, or the fractional increase in the assumed background spontaneous mutation rate 
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of humans." In pursuit of methods to implement the goals of its guidelines, EPA sponsored a 
major effort concerning genetic-risk assessment for the direct-acting mutagen ethylene oxide 
(Dellarco and Farland, 1990; Dellarco et al., 1990; Rhomberg et al., 1990). But EPA does 
not now routinely perform quantitative assessments of genetic risk posed by chemical mutagens 
in the environment as part of any of its regulatory programs. 

EPA's 1986 guidelines are nonspecific not only regarding particular methods to be used by 
the agency for estimating mutagenic risk, but also regarding how such risk might be aggre¬ 
gated with risks estimated for other end points, such as cancer. The suggested measures of 
genetic risk in the guidelines cannot readily be aggregated with EPA's commonly used 
measures of increased cancer risk to individuals or populations. However, individual genetic 
risk could be expressed as increased lifetime risk of expression of a serious inherited genetic 
end point in a person whose parents were both exposed from birth to a given relevant com¬ 
pound at a given effective dose rate. And addition of such a predicted risk to a corresponding 
magnitude of predicted somatic (cancer) risk would be appropriate under assumptions of low- 
dose linearity and independence as discussed above and in Appendix I. 

Risk assessments of ionizing radiation provide precedents for the simple addition of 
quantitative estimates of genetic and cancer risk (e.g., Anspaugh and Robison, 1968; ICRP, 
1977a,b, 1984, 1985). However, EPA has made no systematic effort to consider the combina¬ 
tion of mutagenic and cancer risks. In the context of setting radiological National Emission 
Standards for Hazardous Air Pollutants (NESHAPs), the agency's Office of Radiation 
Programs made a substantial effort to describe quantitative risk estimates for both cancer and 
genetic end points (EPA, 1989b). However, the genetic risk factors were not used later in 
EPA's corresponding quantitative radiologic-risk assessments for radioactive air contaminants 
(EPA, 1989b), nor are they considered in current EPA guidance on how to calculate prelimi¬ 
nary Superfund remediation goals for radionuclides at hazardous-waste sites (EPA, 1991f). 

The importance of considering a quantitative combination of genetic and cancer end points 
depends on the ratio of genetic-to-cancer potency of any given chemical. If the ratio is much 
less than 1, genetic-risk assessment of the chemical is probably unwarranted, because it is 
likely to have little impact on regulatory action. For example, the upper-bound estimate of the 
potency of ethylene oxide (ETO) to produce heritable translocations (HTs) in children of 
exposed men was recently estimated to be equivalent to 0.00066 per part of ETO per million 
parts of air continuously inhaled. This estimate was based on an EPA analysis that applied a 
linearized multistage extrapolation model to dose-response data on HT induction in mice; a 21- 
day critical exposure period was assumed to be potentially damaging to human males (Rhom¬ 
berg et al., 1990). In contrast, EPA had previously estimated ETO’s cancer potency to be 
0.19 per part of ETO per million part of air continuously inhaled over a lifetime—a value 
almost 290 times its estimated HT potency (EPA, 1985c). The genetic risk associated with 
ETO could not therefore constitute a substantial fraction of the genetic-plus-cancer risk unless 
HT represented a very small fraction (e.g., less than 1/290) of all reasonably quantifiable 
ETO-induced genetic end points. This appears to be unlikely, given that HTs constitute 
between about 5% and 10% of CADXMs (ICPEMC, 1983b). 

There are continuing concerns about the adequacy of current approaches (threshold, linear. 
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nonlinear, BD, etc., described in Chapter 4) to characterize the risks associated with potential 
reproductive and developmental hazards (Barnes and Dourson, 1988; Mattison, 1991). 
Particular questions remain regarding thresholds. Although threshold mechanisms might seem 
plausible, the estimation of an upper limit to ensure that doses are safe depends heavily on 
available methods of study and measurement and our knowledge of organ- and tissue-specific 
repair mechanisms. The issue merits continued consideration. This issue is also discussed in 
the NRC report entitled Seafood Safety (NRC, 1991b). 

The current and proposed EPA guidelines concerning reproductive- and developmental- 
toxicity risk assessment are based on the controversial assumption that chemical induction of 
reproductive or developmental toxicity generally has a true or practical threshold dose-response 
relationship. As noted by EPA (1991a), such thresholds might differ among exposed people, 
and EPA has traditionally accommodated such interindividual variability by using an extra 
uncertainty factor or safety factor of 10, whose adequacy remains to be established. 


Measures and Characteristics of Risk 
Overall Characterization Goals 

An essential component of risk characterization is the aggregation of different measures and 
characteristics of risk; the risk assessor must communicate measures and characteristics of 
predicted risk in ways that are useful in risk management. The technical aspects of risk 
aggregation and characterization cannot and should not be separated from the design of useful, 
politically responsible, and legally tenable criteria of risk acceptability, because such criteria 
must generally be based on risk characterizations that follow some standard format, and the 
format must accommodate the criteria. As new, more sophisticated approaches to risk 
assessment and characterization are proposed—such as the incorporation of integrated uncer¬ 
tainty and variability analysis—the corresponding more complicated criteria for risk acceptabil¬ 
ity have not been agreed on. It is therefore appropriate to establish as an interim goal of risk 
characterization the adoption of a format that includes a summary of predicted risk that is 
accurate, comprehensive, easily understood, and responsive to a wide array of public concerns 
about risk. The format should include the magnitude and uncertainty of estimated population 
risk (that is, predicted incidence) as well as individual risk, the uncertainty of estimates of 
costs and competing risks inherent in alternative risk-management options, the degree to which 
estimated risks might vary among exposed individuals, and the time frame of risks imposed. 


Consistency in Characterization: 

Example of Aggregation of Uncertainty 

To the extent that a given aggregated characteristic of a risk assessment, such as uncertainty, 
is addressed in an overall characterization of predicted risk, it should be determined with a 


Chapter 11: Aggregation 


11-10 


Source: https://www.industrydocuments.ucsf.edu/docs/jnflOOOO 


2048282230 


consistent approach to estimates of the magnitudes of the components considered (e.g., 
ambient concentration, uptake, and potency). In the case of uncertainty aggregation, such 
consistency will come about through a rigorous, fully quantitative approach (see Chapter 9). 
But such a fully quantitative approach might be deemed impractical; for example, quanti¬ 
fication of subjective probability judgments in the assessment might be considered difficult or 
misleading. A screening-level alternative to a fully quantitative approach to uncertainty 
aggregation is to use a qualitative or categorical approach that describes, in narrative or tabular 
form, the impact of each component of the analysis on each aspect of predicted risk. 

However, an exclusively qualitative, categorical approach is generally impractical because it 
fails to communicate effectively the fundamental quantitative conclusions of the risk analysis in 
terms that are of direct use to risk managers. 

Thus, the approach to uncertainty aggregation most often used has been a semiquantitative 
approach incorporating specific key assumptions whose merits and impact are discussed 
verbally. The difficulty with this approach lies in ensuring that resulting semiquantitative 
characteristics are properly interpreted and communicated. For example, it would be illogical 
and potentially misleading to characterize a final risk estimate as a "plausible upper bound" on 
risk, if it were derived by aggregating component-specific point estimates that represent a 
mixture of best estimates and statistical upper confidence limits. That is particularly true if the 
components for which best estimates are used are also the components known to be the most 
uncertain among those considered. When, for example, risk is modeled as a simple product of 
estimated quantities (such as concentration, potency, etc.) a great deal of conservatism is lost 
whenever a best estimate is used in place of a far larger corresponding upper-bound value (and 
little conservatism is gained by using an upper-bound value if it is close to the corresponding 
best estimate). Thus, if a semiquantitative approach is to be used, the only way to obtain a 
meaningful "upper-bound" point estimate of risk from component-specific point estimates 
would be to base the "upper-bound" point estimate entirely on "upper-bound" estimates of all 
the component quantities. This point is illustrated by the following example involving EPA's 
cancer-risk guidelines. 

The EPA guidelines for cancer-risk characterize the estimate produced by following the 
guidelines as a "plausible upper bound" on increased cancer risk. Such a risk estimation will 
generally involve a pertinent set of animal bioassay data, an animal-cancer potency estimate, 
and an interspecies dose-scaling factor. According to the 1986 guidelines, the risk assessment 
is to be based on the data showing the most sensitive response (i.e., that give the highest 
estimated potency value or set of related values), and the animal-cancer potency value used is a 
statistical upper confidence limit of potency estimated from the animal-bioassay data set 
selected. The guidelines specify a dose-scaling factor—based on what was intended by EPA to 
be a deliberately conservative assumption that carcinogenic doses are equivalent between 
species if they are expressed as daily mass per unit of body surface area. Recently, EPA 
(1992e) proposed adopting a new scaling factor that is somewhat less conservative because this 
new factor appears to be close to a "best" estimate of what the factor might actually be. 
However, EPA (1992e) noted that 
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"Although scaling doses by [the newly proposed factor] characterizes the trend [relating epidemiologically based 
human-cancer potencies with corresponding experimentally determined ones for animals] fairly well, individual 
chemicals may deviate from this overall pattern by two orders of magnitude or more in either direction. . . .The 
proposed scaling [approach]. . .represents a best guess. . .surrounded by an envelope of considerable 
uncertainty. . . .[It] is intended to be. . .an unbiased projection; i.e. s it is to be thought of as a "best" estimate rather 
than one with some conservatism built in . . .[such] as a "safety factor" or other intentional bias designed to "err on 
the side of safety." 

A similarly large degree of uncertainty associated with interspecies dose scaling was also 
indicated in a recent reassessment of uncertainty pertaining to interspecies extrapolation of 
acute toxicity (Watanabe et al., 1992). Other studies (Raabe et al., 1983; Kaldor et al., 1988; 
Dedrick and Morrison, 1992) provide evidence that a milligram-per-kilogram-per-lifetime dose 
metric may be roughly equivalent across species. These studies compare human carcinogenic¬ 
ity and animal carcinogenicity for alkylating or radioactive agents (administered for therapeutic 
purposes in the case of humans). Dose-scaling uncertainty may thus be substantially far 
greater than that associated with parameter-estimation error for cancer potency in bioassay 
ani m als and be at least as great as that associated with the selection of a bioassay data set for 
analysis. EPA's proposed dose-scaling policy would therefore be an exception to its reason¬ 
ably consistent practice of using component-specific upper bounds when semiquantitative 
aggregation of uncertainty is used to derive a "plausible upper bound” on increased risk. The 
most straightforward way to obtain such an upper-bound dose-scaling factor would be to 
calculate it directly from the best available relevant empirical data that relate epidemiologically 
based human-cancer potencies to corresponding experimentally determined animal-cancer 
potencies (e.g., Raabe et al., 1983; Allen et al., 1988; Kaldor et al., 1988; Dedrick and 
Morrison, 1992). An uncertainty distribution for the scaling factor could also readily be 
developed from these data, and an appropriate summary statistic chosen explicitly from this 
distribution, rather than by fiat and without reference to uncertainty (see, for example, 
Watanabe et al., 1992). 


Uncertainty and Variability 

We have deliberately treated these two concepts separately up to this point in the report, 
because we view them as conceptually quite different even though they share much of the same 
terminology (e.g., "upper confidence limit," "standard deviation"). Indeed, as emphasized in 
Chapters 9 and 10, the realms of uncertainty and variability have fundamentally different 
ramifications for science and judgment: uncertainty forces decision-makers to judge how 
probable it is that risks will be overestimated or underestimated for every member of the 
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exposed population, whereas variability forces them to cope with the certainty that different 
individuals will be subjected to risks both above and below any reference point one chooses. 1 

Thus, any criticism that EPA has assessed or managed a risk too "conservatively" needs to 
consider and explain which type of conservatism is being decried. The use of a plausible but 
highly conservative scientific model, if it imposes large costs on society or the regulated 
community, can throw into question whether it is wise to be "better safe than sorry.” The 
attempt to provide protection to persons at the "conservative" end of a distribution of exposure 
or risk, in contrast, determines who ends up with what degree of safety and thus requires a 
different decision calculus. In particular situations, either uncertainty or variability (or 
perhaps both) might be handled "conservatively." For example, society might in one case 
determine that the marginal costs of protecting individuals with truly unusual hypersusceptibil¬ 
ity were too large relative to the costs of protecting only the majority, but might still choose to 
assess the risk to each group in a highly conservative manner. In another case, society might 
view the central tendency of an uncertain risk as an appropriate summary statistic, yet deem it 
important to extend protection to individuals whose risks are far above the central tendency 
with respect to the varied risks across the population. 

On the other hand, this risk management distinction between uncertainty and variability 
should not blind people to a central fact of environmental health risk assessment: that in 
general, risks are both uncertain and variable simultaneously. In the prototypical hazardous air 
pollutant risk assessment case, one can think of the source exposing each nearby resident to a 
different ambient concentration of each emitted pollutant; each of these concentration values is 
made still more variable by the unique activity patterns, uptake parameters, and susceptibility 
of each individual. Simultaneously, each of these "individualized" parameters is either hard to 
measure or impossible to model with certainty (or both), and all of the "generalized" parame¬ 
ters (such as the inherent carcinogenic potency of each substance) are also surrounded by 
uncertainty. In sum, the source does not impose "a risk”—it imposes a spectrum of individual 
risks, each of which can only be completely described as a probability distribution rather than 
a single number. 

Elsewhere in the report, we have commented on two aspects of the challenge of assessing 
variable and uncertain risks: communicating them correctly and comprehensively (see the 
findings and recommendations for this chapter), and describing how to relate variability to 
uncertainty in order to explicitly target risk management to the desired members of the 
population (average, "high-end," maximally at risk, etc.) in light of the uncertainty (again, see 
the findings and recommendations for this chapter). 


For example, in the 1980s the Consumer Products Safety Commission (CPSC) had to issue a standard regarding how close 
together manufacturers had to place the vertical slats in cribs used by infants, with the aim of minimizing the number of accidental 
strangulations nationwide. Presumably, there was virtually no uncertainty about the diameter of an average infant's head, but there was 
significant variability in distinguishing different infants from each other. CPSC thus had to make a decision about which estimation of 
head size to peg the standard to-an "average" estimate, a 'reasonable worst case,' the smallest (i.e., most conservative) plausible value, 
etc. We suggest that it is not apropos to use the phrase "better safe than sorry" to apply to this kind of reasoning, because uncertainty is 
not at work here. Rather, deciding whether to be conservative in the face of variability rests on a policy judgment about how far to extend 
the attempt to provide safety. 
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Here, we briefly mention two additional complications that arise because uncertainty and 
variability work in tandem. We make no specific recommendations regarding either issue, 
because we feel EPA analysts and other risk assessors need flexibility to account for these 
tpfi hnir al problems as they gradually improve their treatment of the separate phenomena of 
uncertainty and variability. Nevertheless, it is important to keep in mind two other relation¬ 
ships between these phenomena: 

(1) Variability in one quantity can contribute to uncertainty in another. The most relevant 
example of this general phenomenon involves the influence of variability in a quantity on the 
uncertainty in its mean. As mentioned in the introduction to Chapter 10, one way to deal with 
interindividual variability is to substitute the average value of the varying quantity, although 
this does preclude conducting analyses that are meaningful at the individual level. However, 
even this short-cut is not without additional complications, because the new parameter (the 
population average value of the variable quantity) may be rather uncertain if the variability is 
substantial. Although the central limit theorem states that the uncertainty in the mean is 
inversely proportional to the number of observations made, when the quantity varies by orders 
of magnitud e, even "large" data sets (tens or even hundreds of observations) may not be 
sufficient to pin down the mean with the precision desired. A group of 1000 workers observed 
in an epidemiologic study, for example, may have an average susceptibility to cancer signifi¬ 
cantly greater or less than the true mean of the entire population, if by chance (or due to a 
systematic bias) the occupational group has slightly more or slightly fewer outliers (particularly 
those of extremely high susceptibility) than the overall population. In such cases, estimates of 
potency or population incidence drawn from the worker study may be overly "conservative" 

(or insufficiently so). 

(2) The amount of variability is generally itself an uncertain parameter. There are at least 
three factors that work to complicate the estimation of variability. Thus, risk assessment 
parameters that attempt to summarize variability (either as inputs to other calculations or as the 
output for risk management or communication) should be regarded as uncertain unless these 
three factors are deemed unimportant: (1) "double-counting" and overestimation of variability 
may occur when error-prone measurements are made—these errors will tend to make the 
extremes in the population seem more divergent than they truly are; (2) even when measure¬ 
ments are perfect, the amount of variability cannot be perfectly determined from any single 
data set—random parameter uncertainty introduces the possibility that by chance, the popula¬ 
tion observed might be inherently less or more variable than the entire population; and (3) 
there may be "model uncertainty" in deciding what kind of probability distribution to fit to 
variable observations, and hence statistics such as the standard deviation or the upper confi¬ 
dence limit might be in error if they apply to a distribution that does not precisely describe the 
actual variability. 

In sum, EPA should realize that estimates of variability themselves may be too large or too 
small—if "conservatism" is crucial, it may make sense to take account of this impreciseness of 
variability as well as taking account of variability itself (e.g., if fish consumption is deemed to 
be lognormal with a standard deviation somewhere between (x - 6) and (x+5), it might be 
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appropriate to use an upper confidence limit for fish consumption that is in turn based on the 
larger of the two estimates of variability, x+5). 


Aggregation of Uncertainty and Variability 

To the extent that both uncertainty and interindividual variability (that is, heterogeneity or 
differences among people at risk) are addressed quantitatively with separate input components 
(e.g., ambient concentration, uptake, and potency) for aggregation into an assessment of risk, 
the distinction between uncertainty and variability ought to be maintained rigorously through¬ 
out the analytic process, so that uncertainty and variability can be distinctly reflected in 
calculated risk. If no distinction were made between uncertainty-related and heterogene¬ 
ity-related distributions associated with inputs to a given risk calculation, then whatever 
distribution might be obtained as a characteristic of risk would necessarily reflect risk to an 
individual selected at random from the exposed population (Bogen and Spear, 1987). This 
restricted result would render such analyses less useful for environmental regulatory purposes, 
in light of the tendency to focus substantial regulatory attention on increased risk to highly 
sensitive or highly exposed members of the population. 

Another advantage of distinguishing between uncertainty and variability is that it permits 
one to estimate the uncertainty in the risk to the individual who is "average" with respect to all 
characteristics that are heterogeneous among individuals at risk, and the latter risk may be used 
to estimate uncertainty in predicted population risk or number of cases (Bogen and Spear, 
1987). Technical issues that arise in aggregating uncertainty and interindividual variability for 
the purpose of calculating estimated individual and population risk are described in Appendix 
1-3 


Findings and Recommendations 


Multiple Routes of Exposure 

Although the Clean Air Act Amendments of 1990 do not specifically refer to multiple 
exposure pathways, EPA has routinely considered multiple exposure routes in regulatory 
contexts, such as Superfund, that logically concern source-specific pollutants that might 
transfer to other media before human exposure. 

• Health-risk assessments should generally consider all possible routes by which people at 
risk might be exposed, and this should be done universally for compounds regulated by EPA 
under the Clean Air Act Amendments of 1990. The agency's risk-assessment guidance for 
Superfund-related regulatory compliance (EPA, 1989a) can serve as a guide in this regard, but 
EPA should take advantage of new developments and approaches to the analysis of multimedia 
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fete and transport data. This will facilitate systematic consideration of multiroute exposures in 
designing and measuring compliance with Clean Air Act requirements. 


Multiple Compounds and End Points 

When aggregating cancer risk associated with exposures to multiple compounds, EPA adds 
the risk related to each compound in developing its risk estimate. That is appropriate when the 
only risk characterization desired is a point estimate used for screening-level analysis. 
However, if a quantitative uncertainty characterization is desired, simple addition of upper 
confidence limits may not be appropriate. 

• EPA should consider using appropriate statistical (e.g., Monte Carlo) procedures to 
aggregate cancer risks from exposure to multiple compounds if a quantitative uncertainty 
characterization is desired. 

EPA currently uses a specific procedure when analyzing animal bioassay data involving the 
occurrence of multiple tumor types (e.g., lung, stomach, etc.) to estimate the total cancer risk 
associated with exposure to a single compound. In this procedure EPA adds the numbers of 
animals with tumor types that are significantly increased above control levels, such that an 
animal with multiple tumor types counts the same as one with a single tumor type. This 
procedure does not allow full use of the data available and can overestimate or underestimate 
total cancer risk. 

• When analyzing animal bioassay data involving the occurrence of multiple tumor types, 
EPA should use the following default procedure. Cancer potencies should first be separately 
estimated for each tumor type involved with the procedure normally used in the case of 
bioassays involving a single tumor type. The type-specific potencies should then be added as 
upper bounds or using appropriate statistical (e.g., Monte Carlo) methods. This procedure 
should be used unless specific data indicate that occurrence of the different tumor types within 
individual animals are significantly correlated. 


Genetic Effects 

Current EPA guidelines do not clearly state a default option of nonthreshold low-dose 
linearity for genetic effects that can be reasonably estimated for quantitative risk assessment. 

• EPA's guidelines should clearly state a default option of non threshold low-dose linearity 
for genetic effects on which adequate data (e.g., data on chromosomal aberrations or dominant 
or X-linked mutations) might exist. This default option allows a reasonable quantitative 
estimate of, for example, first-generation genetic risk due to environmental chemical exposure. 
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Reproductive and Developmental Toxicants 

While EPA is increasing its use of the benchmark dose, it still uses a threshold model in its 
proposal for regulation for reproductive and developmental toxicants. Although the threshold 
model is generally accepted for these toxicants, it is not known how accurately it predicts 
human risk. Current evidence on some toxicants, most notably lead and alcohol, does not 
unequivocally demonstrate any "safe" threshold and thus has raised concerns that the threshold 
model might only reflect the limits of current scientific knowledge, rather than the limits of 
safety. 

• EPA should continue to collect and use the data needed to evaluate the validity of the 
threshold assumption, and it should make any needed revisions in the proposed model so that 
human risks, particularly those of individuals with above-average sensitivity or susceptibility, 
are accurately estimated. 


"Upper-Bound Estimates'' versus "Best Estimates" 

In a screening-level or semiquantitative risk characterization, component uncertainties 
associated with predicted cancer risk are not generally aggregated in a rigorous quantitative 
fashion. In such cases, it is practical to calculate an "upper-bound" point estimate of risk by 
combining similarly "upper-bound" (and not "best”) point estimates of the component 
quantities involved, particularly for quantities (such as the dose-scaling factor) that are highly 
uncertain. For screening-level analyses, the EPA (1992d) proposal to adopt a new interspecies 
dose-equivalence factor is inconsistent with the 1986 guideline stipulation that risk estimated 
under the guidelines represents a "plausible upper bound" on increased cancer risk, and it is 
inconsistent with the corresponding stipulation that "upper-bound" or health-conservative 
assumptions are to be used at each point in cancer-potency assessment that involves substantial 
scientific uncertainty. 

• For a screening-level or semiquantitative approach in which component uncertainties 
associated with predicted upper-bound cancer risk are not aggregated in a rigorous quantitative 
fashion, the EPA guidelines, to determine upper-bound cancer risk, should require the use of 
an upper-bound (i.e., reasonably health-conservative), rather than a "best," interspecies 
dose-scaling factor consistent with the best available scientific information. 


Uncertainty versus Variability 

A distinction between uncertainty (i.e., degree of potential error) and interindividual 
variability (i.e., population heterogeneity) is generally required if the resulting quantitative risk 
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characterization is to be optimally useful for regulatory purposes, particularly insofar as risk 
characterizations are treated quantitatively. 

• The distinction between uncertainty and individual variability ought to be maintained 
rigorously at the level of separate risk-assessment components (e.g., ambient concentration, 
uptake, and potency) as well as at the level of an integrated risk characterization. 
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Part III 

Implementation of Findings 


The committee believes that a major portion of its charge is to consider how its findings and 
recommendations should be implemented in light of the comprehensive rewriting of Section 
112 by Title HI of the 1990 amendments to the Clean Air Act. Many of the common 
problems in health risk assessment might have arisen because of the two most salient features 
of EPA's implementation of the Red Book paradigm over the last 10 years: the emphasis on 
single outputs of each step, which are then processed into single numbers for risk; and the 
separation of the research and analysis functions into discrete, sequential stages. 

A tiered system of priority-setting would be an important positive development in the 
practice of standard-setting and risk analysis. Currently, standards (goals for achieving health 
and safety) are set in accordance with a Congressional mandate to provide "an adequate margin 
of safety.” Where data do not exist (particularly with respect to responses to low doses and 
mechanisms of toxicity), EPA has generally chosen default options that, in addition to being in 
keeping with current scientific knowledge, are intended to be conservative (i.e., health- 
protective) in the outcomes to which they lead. This protective approach provides the basis for 
developing a stepwise, tiered system for assigning priorities to chemicals to be examined for 
potential regulation. As a first tier—usually in the absence of data—computations can be made 
(with the appropriate default assumptions) that lead to a possible regulatory standard. If this 
standard is readily achievable, no further analysis is called for. If the standard is not achiev¬ 
able, data will be sought to replace the possibly too-conservative default assumptions. 

Substituting more chemical-specific information for default assumptions will usually lead to 
less rigid and thus more easily attainable standards (or higher "safe" doses). (The rare 
situations in which this relaxation of the standards does not occur would imply that the default 
assumptions were not sufficiently health-protective and so needed to be re-examined.) 

A stepwise process that replaces default assumptions with specific data can be expected to 
yield more and more firmly established standards (regulatory doses); i.e., uncertainty should 
be reduced as a consequence of having more information. The tiered process for setting 
standards thus reflects the philosophical process of proceeding from conjecture ("it is reason¬ 
able that. . .") through information to (one hopes) wisdom. 

The issue of implementation is discussed in Chapter 12, the final chapter, from two points 
of view. First, technical guidance is provided on EPA's implementation of the recommenda¬ 
tions in a regulatory context. Second, the committee discusses institutional issues in risk 
assessment and risk management. M 
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Implementation 


Health risk assessment is one element of most environmental decision-making—a component of 
decisions about whether, how, and to what degree the assessed risk requires reduction. The 
factors that may be considered by decision-makers depend on the requirements of applicable 
statutes, precedents established within the responsible government agencies, and good public 
policy. This chapter discusses how the risk-assessment recommendations in this report could 
be implemented in the context of Section 112 of the Clean Air Act (as amended in 1990), and 
it discusses several institutional issues in risk assessment and risk management. 


Priority-Setting and Section 112 

As we explained in Chapter 2, Section 112 calls for EPA to regulate hazardous air pollutants 
in two stages. In the first, sources will be required to do what is feasible to reduce emissions. 
In the second, EPA must set "residual-risk" standards to protect public health with an ample 
margin of safety if it concludes that implementation of the first stage of standards does not 
provide such a margin of safety. This second stage will require use of risk assessment. 

Neither the resources nor the scientific data exist to perform a full-scale risk assessment on 
each of the 189 chemicals listed as hazardous air pollutants by Section 112. Nor, as we 
noted in Part n, is such an assessment needed in many cases. 

We therefore urge an iterative approach to risk assessment. Such an approach would start 
with relatively inexpensive screening techniques and move to more resource-intensive levels of 
data-gathering, model construction, and model application as the particular situation war¬ 
ranted. To guard against the possibility of underestimating risk, screening techniques must be 
constructed so as to err on the side of caution when there is uncertainty. The results of these 
techniques should be used to set priorities for the gathering of further data and the application 
of successively more complex techniques. These techniques should then be used to the extent 
necessary to make a judgment. The result would be a process that supports the risk- manage¬ 
ment decisions required by the Clean Air Act and that provides incentives for further research, 
without the need for costly case-by-case evaluations of individual chemicals. 

Under an iterative approach, a screening analysis is followed by increases in the refinement 
of the estimate, as appropriate. In effect, each iteration amounts to a more detailed screen. 

As we have explained in Chapter 6, screening analyses need to incorporate conservative 
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assumptions to preclude the possibility that a pollutant that poses dangers to health or 
welfare will not receive full scrutiny. 

Considering the effort required to carry out a "full-scale" risk assessment of 189 potentially 
hazardous substances and the current resources of the agency, it is unlikely that this task can 
be accomplished within the time permitted by the act if full-scale risk assessments must be 
conducted by EPA itself. This committee recommends a priority-setting scheme (as described 
in the following sections) based on initial assessments of each chemical's possible impact on 
human health and welfare. But Congress should recognize that the resources now available to 
EPA probably will not support a full-scale risk analysis for each source or even each source 
category within the time permitted, even with priority-setting. Thus, EPA will need alterna¬ 
tives to full-scale risk assessment, and attention should be given to setting priorities for the 
allocation of resources. In addition, a full statement of resource requirements should be 
developed and presented to Congress for its use in decisions about budget and for its under¬ 
standing and guidance with regard to reducing the task. 


Iterative Risk Assessment 

To implement Section 112, the committee generally supports the tiered, iterative risk- 
assessment process proposed by EPA in its draft document as shown in Appendix J. As stated 
by EPA, this process is based on the concept that as the comprehensiveness of a risk assess¬ 
ment increases, the uncertainty in the assessment decreases. 

In the absence of sufficient data or resources to characterize each risk-assessment parameter 
accurately, EPA deliberately uses default options that are intended to yield health-protective 
risk estimates. Lower-tier risk assessments that are used for preliminary screening rely heavily 
on default options, and their results should be health-protective. If a lower-tier risk assessment 
indicates that an unacceptable health risk could be associated with a particular exposure and a 
regulated party believes that the risk has been overestimated, a higher-tier risk assessment can 
be performed. The higher-tier risk assessment would be based on more precise (and less 
uncertain) exposure and health information instead of relying on the default options. Con¬ 
versely, if EPA believes that a lower-tier risk assessment has underestimated the health risk 
associated with a particular exposure, a higher-tier risk assessment might yield a more reliable 
estimate. 

The following sections evaluate each step in the health-risk assessment process with 
reference to how EPA plans to implement its tiered approach. 


Exposure Assessment 

EPA (1992f) has proposed a tiered scheme for using health risk assessments to delist source 
categories and eliminate residual risk. EPA asserts that this scheme provides health-protective 
estimates of risk by assuming maximal exposure levels, except for cases related to complex 
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terrains (for which an alternative dispersion model should be selected from the complex-terrain 
models available to EPA to estimate maximal concentrations of chemicals in air and hence 
maximal exposure levels). 

In the initial step of the tiered approach (see Table 12-1), the emission rate for a facility is 
multiplied by a dispersion value obtained from a table and chosen on the basis of two site- 
specific parameters: stack height and the approximate distance to the site boundary line. A 
generic "worst-case" meteorology applicable to all noncomplex terrain is used to obtain the 
dispersion factors for a simple Gaussian-plume model with worst-case plant parameters (e.g., 
zero-buoyancy plume and zero exit velocity). 

The second tier uses a simple, single Gaussian-plume model that incorporates site-specific 
data on the site boundary distance; the stack height, exit velocity, temperature, and diameter; 
the urban-rural classification; and the building dimensions. Again, a generic worst-case 
meteorology is used in the calculation. 

In the third tier, the modeling would include multiple-point release, local meteorologic 
characteristics, and the choice of specific local receptor-site locations. The maximal exposure 
is calculated by multiplying the estimated concentration by residence time. EPA is debating 
the extent to which it will use less than lifetime residence (i.e., alter the 70-year-lifetime 
assumption). 

In a presentation made to the committee by EPA staff, a fourth tier was described that 
would incorporate time-activity modeling, as in the Human Exposure Model II (HEM II). 
HEM-H uses an approach similar to that of the National Ambient Air Quality Standards 
(NAAQS) Exposure Model (NEM), which has been used in exposure assessments for criteria 
pollutants (tropospheric ozone, sulfur dioxide, etc.). However, the NEM has not been fully 
evaluated and validated (NRC, 1991a). 

There are a number of difficulties associated with the method in the initial tiers. First, EPA 
does not specify that a conservative emission rate should be used; it will use the emission rate 
for normal operation of a plant at full capacity. In addition, none of EPA's current emission 
estimation methods accounts for "upset” situations with higher than normal emissions or for 
the emission-estimate uncertainty. Therefore, current emission estimates cannot be relied on 
as necessarily conservative. 

Second, the committee reiterates its earlier concern (see Chapter 7) about the use of the 
Gaussian-plume model beyond the lower-tier screening level. Even there, complex terrain can 
create substantial problems. The EPA complex-terrain models have focused on emissions 
released from tall stacks toward the side of a hill or valley, and not on poor dispersion of 
material from a point or area source within a valley. Models for complex terrain have been 
developed and evaluated by the atmospheric-research community. The committee does not 
recommend any specific model, but suggests that EPA look beyond its set of existing models 
to find the best possible ones for the dispersion of hazardous air pollutants in the particular 
type of complex terrain that applies in each case. In addition, models should be considered 
that account for the possibility of a negative buoyancy plume (i.e., gas heavier than air). 
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Table 12-1. Summary of EPA's draft tiered risk-assessment approach as presented to the committee* 

Tier 1: Lookup Tables 

• Two tables: short- and long-term (based on EPA's SCREEN model) ] 

• Inputs: emissions rate, release height, fenceline distance 

• Outputs: Maximum offsite concentration (focus on maximum exposed individual, MEI) 

Maximum offsite cancer risk (based on unit risk estimate, URE) 

Chronic noncancer hazard index (based on chronic health thresholds) 

Acute noncancer hazard index (based on acute health thresholds) 

Tier 2: Screening Dispersion 

• Based on EPA's SCREEN model (uses conversion factor for long-term) 

• Inputs: Tier 1 + stack diameter, exit velocity and temperature, rural/urban classification, and building 
dimensions 

• Outputs: 

Maximum offsite concentration and downwind distance (focus on MEI) 

Cancer risk and/or noncancer hazard index 

Tier 3: Site-Specific Dispersion Model 

• Based on EPA's TOXLT, TOXST models (uses the ISC dispersion model) 

• Inputs: Tier 1 + Tier 2 + local meteorology, release point and fenceline layout, terrain features, release j 

frequency, and duration 

• Outputs: 

Long-term - receptor-specific risk, chronic noncancer hazard index (MEI) | 

Short-term - receptor-specific hazard index exceedance rate (MEI) I 

• Ambient monitoring used to enhance modeling or as alternative on case-by-case basis for difficult modeling j 
applications 

Tier 4: Site-Specific Dispersion and Exposure Model i 

• Based on EPA's HEM II model ! 

• Inputs: Tier 3 -4* population model j 

• Outputs: Maximum offsite concentration (MEI), exposure distribution, and population risk (incidence) with j 

optional characterization of uncertainties j 

• Personal monitoring used as alterative on case-by-case basis for difficult modeling applications 

Notes: (1) Approach considers flat or rolling terrain only; j 

(2) Complex terrain alternatives used on case-by-case basis; s 

(3) Analysis considers only direct inhalation exposure; | 

(4) Tiers proceed from most conservative and least data intensive (Tier 1) to least conservative and most data i 

intensive (Tier 4). ! 

j 

Source: Guinnup, 1992 (see Appendix J). 
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health agencies, and by other parties that look to EPA to provide current information on 
chemical toxicity. 

The IRIS database will be the primary source of toxicity data for the tiered risk-assessment 
approach described here. The committee believes that it is appropriate for EPA to use IRIS as 
its preferred data source for toxicity information, rather than duplicate the effort needed to 
assemble and maintain such information for those of the 189 chemicals specified in Section 
112 that are in IRIS. For chemicals that require a higher-tier risk assessment, EPA could 
supplement the information in IRIS with additional data, probability distributions, and 
modeling approaches. For Section 112 chemicals not yet in IRIS, EPA must collect and enter 
data on carcinogenic and noncarcinogenic effects. 

For many of the 189 chemicals now on the Section 112 list, there are no IRIS entries, or the 
existing entries do not include cancer potencies for suspected carcinogens or RfCs for chemi¬ 
cals suspected of causing acute or chronic noncarcinogenic health effects. In these cases, it 
will be appropriate for EPA to develop crude screening estimates of cancer potencies and RfCs 
for use in research planning; if the screening values are entered in IRIS, they should be clearly 
identified as screening values. These estimates should be combined with exposure estimates to 
calculate potential cancer risks and the likelihood of acute and chronic noncancer health 
effects. Such estimates may be based, for example, on in vitro tests for carcinogenicity, 
expert judgment on structure-activity relationships, and other available information and 
judgment on the toxicity of the chemical in question. These crude estimates should not be 
used as a basis for regulatory decisions when the supporting data are not adequate for such use. 
However, an entry can and should summarize current information on the extent to which a 
chemical might be a potentially important threat to public health. If a bioassay of the chemical 
is under way through the National Toxicology Program or elsewhere, the estimated date of 
availability of results should be stated in IRIS. 

A review of IRIS by the EPA Science Advisory Board (SAB) noted the importance of IRIS 
for both EPA and non-EPA users (Appendix K). If IRIS entries are to be used for risk 
assessments that lead to major risk-management decisions, then EPA must ensure their quality 
and keep them up to date. It is EPA's standard practice that IRIS files must be assessed in 
their entirety so that cancer potencies and RfCs are not distributed without an accompanying 
narrative description of their scientific basis; IRIS is intended not only as a source of numeri¬ 
cal data, but also as an important source of qualitative risk-assessment information. The 
appropriate caveats and explanations of numerical values are important for keeping risk 
managers and other IRIS users fully informed about health-risk information. 

The SAB noted that chemical-specific risk assessments, such as Health Assessment 
Documents (HADs) and SAB reviews of HADs, should be referenced and summarized in 
IRIS. Where different risk assessments have yielded different cancer potencies or RfCs, the 
file should include an explanation that relates these differences to variations in data, assump¬ 
tions, or modeling approaches. Data deficiencies and weaknesses in a risk assessment that 
might be remedied through further data collection and research should be described in the file. 
In this way, IRIS can evolve into a high-quality information support system for the needs of 
EPA and other users relative to the Section 112 chemicals, providing not just one set of 
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For the conditions under which hazardous air pollutants are emitted from many emission 
points within a plant, EPA has not demonstrated that the simple, single Gaussian-plume 
approach (choosing dispersion values from a table generated on the basis of a generic worst- 
case meteorology and worst-case plant-dispersion characteristics) will be appropriate for all the 
situations to which it might be applied. The Gaussian-plume models have been tested for the 
dispersion of criteria pollutants from point sources that typically have good dispersive 
characteristics (e.g., tall stacks, high thermal buoyancy, and high exit velocity). However, it 
has not been demonstrated that this generic worst-case meteorology is fully representative of 
any location, such as cities with substantial local perturbations in the dispersion characteristics 
(surface roughness, street canyon, heat-island effects, etc.). The committee recommends that, 
until the evaluations can be completed, exposure assessment for source delisting and evaluating 
residual risk begin at EPA's current Tier 3, where the industrial source complex (ISC) model 
with local meteorology and local receptor-site choices will provide better estimates of the 
worst-case possibilities. If Tiers 1 and 2 can be shown definitively to estimate exposure 
conservatively, they could be incorporated into the delisting, priority-setting, and residual-risk 
process. 

In accordance with the discussion in Chapter 7, the committee recommends that distributions 
of pollutant concentration values be estimated with available evaluated stochastic dispersion 
models that provide more realistic descriptions of the atmospheric dispersion process and that 
incorporate variability and uncertainty in their estimates. If the screening process suggests that 
a source cannot be excluded from further review, exposure estimation should be more 
comprehensive and incorporate more advanced methods of emission characterization, sto¬ 
chastic modeling of dispersion, and time-activity patterns, as discussed in Chapter 7. 

Exposure assessment can be improved as necessary by incorporating more explicit local 
topographic, meteorologic, and other site-specific characteristics. However, if the regulated 
sources find it acceptable to be regulated on the basis of a (truly conservative) screening 
analysis, then there should be no obligation to go further. If they are not content, then the 
sources should bear the burden of doing the higher-tier analysis, subject to EPA guidelines and 
review. 


Assessment of Toxicity 

In EPA’s proposed approach, four metrics will be used to determine whether the predicted 
impact of a source should warrant concern: lifetime cancer risk, chronic noncancer hazard 
index, acute noncancer hazard index, and frequency with which acute hazard index is ex¬ 
ceeded. The toxicity data needed to evaluate these metrics, such as weight-of-evidence 
characterizations and cancer potencies for carcinogenicity and reference concentrations (RfCs) 
for noncancer end points, can be found by referring to the Integrated Risk Information System 
(IRIS) on-line database (reference to IRIS documentation; cf. Appendix J, p. 4). This database 
is maintained by EPA's Environmental Criteria and Assessment Office within the Office of m 
R esearch and Development for use by EPA's various program offices, by state air-quality and 

t-J 
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numbers for dose-response assessment, but also a summary of alternative approaches, their 
strengths and weaknesses, and opportunities for further research that could improve risk 
estimates. 


Summary 

The committee supports EPA's general concept of tiered risk assessment with two modifica¬ 
tions. First, the tiered approach requires a conservative first level of analysis. EPA asserts 
that its approach provides a conservative risk estimate, except in the case of complex terrain. 
But EPA has not yet demonstrated that this assertion is valid. Second, rather than stopping a 
risk assessment at a particular point, EPA should encourage and support an iterative risk- 
assessment process wherein improvements in the accuracy of the risk estimate will replace the 
initial screening estimate. This process will continue until one of three possible conclusions is 
reached: (1) the risk, assessed conservatively, is found to be lower than the applicable 
decision level (e.g., 1 in a million excess lifetime risk of cancer); (2) further improvements in 
the model or. data would not significantly change the risk estimate; or (3) the source or source 
category determines that the cost of reducing emissions of this pollutant are not high enough 
for it to justify the investment in research required for further improvements in the accuracy 
and precision of analysis. This procedure provides private parties with the opportunity to 
improve the models and data used in the analysis. 

EPA must avoid interminable analysis. At some point, the risk-assessment portion of a 
decision should end, and a decision should be made. Reasonable limits on time (consistent 
with statutory time limits) and resources must be set for this effort, and they should be based 
on a combination of the regulatory constraints and the benefits gained from additional scien¬ 
tific analysis. It is not necessary to determine or measure every variable to high accuracy in 
the risk-assessment process. Rather, the uncertainties that have the most influence on a risk 
assessment should be the ones that the risk assessor most seeks to quantify and then reduce. 


Epa Practices: Points to Consider 

The committee throughout this report has noted differences between the methods EPA is 
currently using and practices the committee considers useful in the risk-assessment process. 
The committee's recommendations (summarized below) highlight differences that should be 
considered in the process of EPA's undertaking its proposed tiered risk assessment approach. 

• Select and validate an appropriate emission and exposure-assessment model for each given 
implementation in the risk-assessment process. 

• Use a carcinogen-classification scheme that reflects the strength and relevance of evidence 
as a supplement to the proposed narrative description. 

• Screen the 189 chemicals for programmatic priorities for the assessment of health risks, 
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identify gaps in the data on the 189 chemicals, develop incentives to expedite generation of the 
needed data, and evaluate the quality of data before their use. 

• Clarify defaults and the rationales for them, including defaults now "hidden," and develop 
criteria for selecting and departing from the defaults. 

• Clarify the sources and magnitudes of uncertainties in risk assessment. 

• Develop a default factor or procedure to account for the differences in susceptibility 
among humans. 

• Use a specific conservative mathematical estimation technique to determine exposure 
variability. 

• Conduct pediatric risk assessments whenever children might be at greater risk than adults. 

• Evaluate all routes of exposure to address multimedia issues. 

• Use an upper-bound interspecies dose-scaling factor for screening-level estimates. 

• Fully communicate to the public each risk estimate, the uncertainty in the risk estimates, 
and the degree of protection. 


Implications for Priority-Setting for Title HI Activities 

With a large number of hazardous air pollutants, hundreds of source categories, and perhaps 
hundreds of sources within many of those categories, and with strains on personnel and 
financial resources, EPA will need to set priorities on its actions under Section 112. In 
addition, Title IX of the Amendments requires EPA to perform health assessments at a rate 
sufficient to make them available when needed for the residual risk assessments under Title III 
(approximately 15 per year). To respond to these requirements, EPA will have to determine 
data needs, the level of analysis needed, and the criteria for determining priorities under the 
Clean Air Act, as well as seek sufficient funds for conducting these analyses. 

It is important that EPA establish priorities for its risk assessment activities. In the past, 
EPA has often appeared to base its priorities on the ease of obtaining data on a particular 
chemical. Rather, EPA should acknowledge the relevance and strength of the existing data on 
each of the 189 chemicals (and mixtures) on the list, identify the gaps in scientific knowledge, 
and set priorities for filling the gaps so that research that is likely to contribute the most 
relevant information in the most time- and cost-effective manner will be conducted first. 

At a minimum, an inventory of the relevant chemical, toxicologic, clinical, and epidemio¬ 
logic literature should be compiled for each of the 189 chemicals (or mixtures). For each 
chemical without animal test data, a structure-activity evaluation should be conducted; and for 
each mixture, results of available short-term toxicity tests should be analyzed. If the evidence 
from this step or from reviews of the clinical, epidemiologic, or toxicologic literature suggests 
potential human health concerns, aggregate emission data and estimates of potentially exposed 
populations should be reviewed. The completed preliminary analyses, including a description 
of the assessment process used and the findings, should be placed in the public domain (e.g., 
IRIS or another mechanism readily accessible to the public). The inclusion of exposure data 
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would represent a departure from past practices, and the database might need to be restructured 
to accommodate this new information. 

For any chemical (or mixture) for which preliminary results suggest a potential health 
concern, it is appropriate to use more accurate emission data (including existing source-specific 
data), information on the environmental fate and transport of the chemical (or mixture), and 
more accurate characterizations (e.g., types and estimated numbers) of the populations that 
may be at risk of exposure, including potentially sensitive subpopulations such as children and 
pregnant women. In addition, a more intensive review of the relevance and strength of the 
available animal and human evidence (including toxicologic, clinical, and epidemiologic) data 
should be developed to refine insights into the probable human-health end points. If the 
evidence on a chemical (or mixture) and exposure still suggests potential human health effects, 
the agency should conduct a comprehensive risk assessment. This assessment should be 
conducted and communicated in accord with the recommendations elsewhere in this report, 
and the limitations of the data and the related assumptions, limitations, uncertainties, and 
variability should be appropriately stated with the final output of the assessment. 

In summary, this iterative approach to gathering and evaluating the existing evidence is 
intended to produce a risk assessment for each of the 189 chemicals (or mixtures) that is 
appropriate to the quality and quantity of available evidence, the estimated size of the problem, 
and the most realistic scientific judgment of potential human-health risks based on that 
evidence. The committee believes that the process will result in a time- and cost-efficient 
mechanism that will effectively set priorities among the 189 chemicals (or mixtures) that fit the 
probable public-health concerns about them. 


Model Evaluation and Data Quality 

Data should not be used unless they are explicitly judged to be of sufficiently high quality 
for use in an activity as sensitive as risk analysis. No data should be incorporated into the 
risk-assessment process unless the method used to generate them has been peer-reviewed 
before its use. Table 12-2 indicates some steps that EPA could take to substantiate and 
validate its models and assumptions before use. 

EPA should take additional steps to ensure that methods used to generate data for risk 
assessments are scientifically valid, perhaps through the use of its Science Advisory Board or 
other advisory mechanisms. A process for public review and comment, with a requirement for 
EPA to respond, should be available so that industry, environmental groups, or the general 
public may raise questions regarding the scientific basis of a decision made by EPA on the 
basis of its risk-assessment process. 
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Table 12-2. Example of Procedure for Methods, Data, and Model Evaluation 
Database Evaluation and Validation 

1. Develop data-quality guidelines that require all data submitted to agency to meet minimal quality level relative 
to their intended use before use in given risk-assessment tier. 

2. Conduct critical review of data-gathering and data-management systems to ensure that quality and quantity of 
Aita are sufficient to meet EPA’s risk-assessment responsibilities under act. 

3. Document procedures used to develop data, including why particular analytic or measurement method was 
chosen and its limitations (e.g., sources of error, precision, accuracy, and detection limits). 

4. Characterize and document data quality by indicating overall robustness, spatial and temporal 
representativeness, and degree of quality control implemented; define and display accuracy and precision of 
measurements; indicate how missing information is treated; identify outliers in data. 

5. Account for uncertainty and variability in collection and analysis of data. 

Model Evaluation and Validation 

1. Develop model-validation guidelines that indicate minimal quality of model that can be used for given risk- 
assessment purpose. 

2. Conduct critical review of each model used in risk-assessment process to ensure that quality and quantity of 
output of each model are sufficient to meet EPA's risk-assessment responsibilities under act. 

3. Assess database and establish and document its appropriateness for model selected. 

4. Conduct sensitivity testing to identify important input-controlling parameters. 

5. Assess accuracy and predictive power of model. 


Default Options 

We have noted in previous chapters that EPA should articulate more explicit criteria by 
which it will decide whether it is appropriate to use an alternative to a default in risk assess¬ 
ment. Such criteria may be expressed either in the form of a general standard or in terms of 
specific types of evidence that the agency considers acceptable. 

Critics of EPA*s use of defaults have characterized the issue of their scientific validity in 
binary terms: either they are supported by science, in which case they are deemed legitimate, 
or they are contradicted by new knowledge, in which case they might be too conservative or 
not sufficiently protective. The reality that EPA confronts is more complex than that dichot¬ 
omy. New scientific knowledge is rarely conclusive at its first appearance and rarely gains 
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acceptance overnight. Rather, evidence accumulates, and its validity and weight are gradually 
established through a transition period. The challenge for EPA is to decide when in the course 
of this evolutionary development the evidence has become strong enough to justify overriding 
or supplementing an existing default assumption. 

Management considerations can appropriately be permitted to influence science-policy 
decisions related to deviations from established default positions. The committee emphasizes 
the desirability of well-articulated criteria for deviation from defaults. If new scientific 
evidence suggests that a supposedly conservative default option is not as conservative as 
previously believed, a new default option might be substituted. EPA needs a procedural 
mechanism that will allow departure from existing default models and assumptions. A more 
formal process should be developed. 


Uncertainty Analysis 

Not characterizing the uncertainty in an analysis can lead to inappropriate decisions. In 
addition, attempting to incorporate default assumptions of unknown conservatism into each 
step of a risk assessment can lead to an insufficiently or too conservative analysis. 

The committee believes that the uncertainty on a risk (i.e., risk characterization) can be 
handled in three ways: 

(1) Conduct a conservative screening analysis. 

(2) Conduct a generic uncertainty analysis. 

(3) Conduct testing or analysis to develop plant-specific and chemical-specific probability 
distributions. 

A possible uncertainty-analysis process is described in Table 12-3. As stated earlier, a key 
factor in deciding to increase the scope and depth of uncertainty analysis should be the extent 
to which expected costs and risks might alter decisions. 

For parameter uncertainty, enough objective probability data are available in some cases to 
permit estimation of the probability distribution. In other cases, subjective probabilities might 
be needed. For example, a committee might conclude on the basis of engineering judgment 
that emission estimates calculated with emission factors are likely to be correct to within a 
factor of 100 (see discussion in Chapter 7) and be approximately lognormally distributed. 
Thus, the median of the estimated distribution would be set equal to the observed or modeled 
emission estimate, and the geometric standard deviation would be taken as approximately 10. 
If making such a generic-uncertainty assumption and then picking a conservative estimator 
from the distribution leads to an estimate that is above the relevant decision-making threshold, 
that should govern the decision unless affected parties wish to devote more resources to 
improving the risk characterization. If the risk characterization is sufficient for decision¬ 
making purposes, then it will not be necessary to improve it. 
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Table 12-3. Example of Procedure for Uncertainty Analysis 
Preliminary Steps 

1. Conduct generic review of each parameter in each step of risk-assessment process and determine default 
distribution on the basis of objective probabilities, if possible, or subjective probabilities, if sufficient information 
is not available. If subjective probabilities cannot be used because of lack of consensus, assume either continuous- 
uniform or discontinuous-dichotomous distribution between reasonable lower and upper bounds or if unimodahty is 
reasonable assumption, triangular distribution (between reasonable upper and lower bounds) or lognormal 
distribution (with reasonable estimates of geometric mean and standard deviation). 

Improving Generic Uncertainty Analysis 

2. If it is decided that default uncertainty analysis should be improved, conduct review of default probability 
distribution for each parameter to determine whether default distribution is reasonable. If this distribution is not 
reasonable, conduct step 3 by either of two methods:. 


3a. Conduct sensitivity analysis by replacing each 
component distribution with its corresponding 
mean. 

3b. Select probability distribution for most-sensitive 
parameters on basis of experience, judgment, and 
available information from existing data samples, 
parameter-value ranges, most likely values, or 
range of most likely values. 


3a. Conduct sensitivity analysis by identifying most 
influential parameters for each model component 
(sensitivity index = change in model result per unit 
change in value of parameter) 

3b. Determine uncertainty in each parameter—e.g., 
uncertainty index = ratio of standard deviation of 
parameter X to mean value of parameter X , or 
number of orders of magnitude of uncertainty = 
log(ratio of upper-bound order statistic 
corresponding lower-bound order statistic). 

3c. Determine model sensitivity-uncertainty index by 
multiplying sensitivity index for each parameter b>j 
its uncertainty index. 

3d. Select probability distribution for only influential 
parameters on basis of experience, judgment, and 
available information from existing data samples, i 
parameter-value ranges, most likely values, or j 
range of most likely values. j 
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Table 12-3 (Continued), Example of Procedure for Uncertainty Analysis 

Uncertainty Analysis Flowchart 

4. Conduct a Monte Carlo analysis by using probability distributions of parameters as input for simplified versions 
of each model (e.g., emissions, exposure, and dose-response relationship) to generate a set of synthetic (Monte 
Carlo) probability distributions for output from each model. Approaches other than Monte Carlo might be equally 
feasible. 

5. For each plausible scientific model (i.e., the default plus any plausible alternatives), conduct a numerical 
analysis (e.g., Monte Carlo) to determine the probability-distribution function of risk due to uncertainty in the 
parameters selected in Step 4. Present each distribution separately or combine them into a single representation, 
clearly indicating which portions of the distribution are derived from fundamental controversy about which model 
might be correct. 

6. Conduct a "reality check" to ensure that resulting risk-estimate distribution makes scientific sense; if not, 

adjustments may be made. Objective of this analysis is to improve representation of uncertainty. Clearly state that ! 
uncertainty representation does not characterize all uncertainty associated with estimated risk. [ 

i 

i 

7. Repeat analysis for each type of risk measurement(e„g., individual risk, population risk, and years of life lost) i 

needed for decision-making. ! 

I 

Risk Management j 

8. Judge what probability provides sufficient level of confidence relative to regulatory decision needed. For I 

example, risk manager might judge mean of upper 5 % of distribution to be point estimate that is appropriately j 

"conservative" within context of regulatory decision. This is simple way to guarantee modest but tangible amount | 
of conservatism with respect to both average and upper tail of uncertainty distribution. j 


Institutional Issues in 
Risk Assessment and Management 

EPA's conduct of risk assessment has been evaluated in previous chapters largely from a 
technical perspective, with the aim of increasing the scientific reliability and credibility of the 
process. But EPA operates in a decision-making context that imposes pressures on the conduct 
of risk assessment, and these contextual pressures have led to recurrent problems of scientific 
credibility, the most important of which were noted in Chapter 2. 

Criticisms of EPA's risk assessments take a variety of forms, but many of them focus on 
three basic decision-making structural and functional problems: unjustified conservatism, 
often manifested as unwillingness to accept new data or abandon default options; undue 
reliance on point estimates generated by risk assessment; and a lack of conservatism due to 
failure to accommodate such issues as synergism, human variability, unusual exposure 
conditions, and ad hoc departures from established procedures. Although some of those 
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criticisms might have been overstated (and we provide evidence in earlier chapters that they 
might have been), it is important for EPA to understand the features of its internal organiza¬ 
tion, decision-making practices, and interactions with other federal agencies that lead to these 
criticisms of its performance. The agency's prevailing assumptions concerning the appropriate 
role of risk assessment and its relationship to risk management also should be re-examined. 


Stability and Change 

Tike any other complex organization, EPA is subject to many competing institutional 
pressures that affect the quality and credibility of its decisions. The agency is expected to use 
the best possible science in risk assessment; yet assessments must often be carried out under 
conditions that preclude deliberation or continued study. Problems of intra-agency coordina¬ 
tion that have persisted throughout EPA's history create communication gaps between risk 
assessors and managers. The firefighting mode in which the agency all too often operates 
hinders the design of effective long-range research programs and even the formulation of the 
right questions for science to answer. As in all bureaucracies, it often seems safest to take 
refuge in established approaches, even if these have begun to appear scientifically outdated. 
External pressures, such as the demands of state agencies for precise guidance, strengthen this 
tendency. 

These overarching managerial problems are faced by any regulatory body that is responsible 
for rendering consistent decisions based on changing scientific knowledge. Uncertainty, 
variability, and imperfections in knowledge make it difficult to control environmental risks. To 
remain accountable to the public under these circumstances, regulatory agencies like EPA must 
assess uncertain science in accordance with principles that are fully and openly articulated and 
applied in a predictable and consistent manner from case to case. Risk-assessment guidelines 
and default assumptions were designed to accomplish those objectives, and they have suc¬ 
ceeded to a large extent in making EPA's decisions both transparent and predictable. 

But an unintended side effect of such explicit decision-making rules is that they can run the 
risk of becoming rigid over time to the detriment of scientific credibility. Science-policy rules 
might ensure a valuable degree of consistency from one case to another, but they do so in part 
by sometimes failing to stay abreast of changing consensus in the scientific community. Some 
have criticized EPA for allowing bureaucratic considerations of consistency to override good 
scientific judgment. In trying to ensure that like cases are treated alike, the agency might fail 
to acknowledge, or even recognize, the scientific reasons why a new case is substantially 
unlike others in ostensibly the same category. In short, risk-assessment guidelines can be 
applied in practice like unchangeable rules. That is unfortunate, as articulated earlier in the 
discussion on guidelines versus requirements. 

Since the mid-1970s, numerous reports and proposals have addressed the generic problem of 
enlisting the best possible science for EPA's decision-making. We note, for example, a 
January 1992 report, Safeguarding the Future (EPA, 1992f), submitted to the EPA administra¬ 
tor and containing detailed recommendations for strengthening EPA's scientific capabilities. 
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Such reports have stressed the need for high-quality scientific advice, expanded peer review, 
and adequate incentives for staff scientists—clearly important issues that have attracted 
attention at the highest levels of EPA's administration, but have not been effectively imple¬ 
mented. The agency's decision-making practices have evolved since the mid-1970s, defining a 
positive, although gradual, learning curve. There can be little doubt that EPA is aware, at a 
conceptual level, of steps that can be taken to improve both its in-house scientific capabilities 
and its collaboration with the independent scientific community. 


Management As Guide To Assessment 

A more subtle and less widely recognized impediment to good decision-making on risk 
arises from a rigid adherence to the principle of separating risk assessment from risk manage¬ 
ment. The call to keep these two functions distinct was originally articulated in response to a 
widespread perception that EPA was making judgments on the risk posed by a particular 
substance not on the basis of science, but rather on the basis of its willingness to regulate the 
substance. The purpose of separation, however, was not to prevent any exercise of policy 
judgment at all when evaluating science or to prevent risk managers from influencing the type 
of information that assessors would collect, analyze, or present. Indeed, the Red Book made it 
clear that judgment (also referred to as risk-assessment policy or science policy) would be 
required even during the phase of risk assessment. The present committee concludes further 
that the science-policy judgments that EPA makes in the course of risk assessment would be 
improved if they were more clearly informed by the agency’s priorities and goals in risk 
management. Protecting the integrity of the risk assessment, while building more productive 
linkages to make risk assessment more accurate and relevant to risk management, will be 
essential as the agency proceeds to regulate the residual risks of hazardous air pollutants. 

Risk assessment should be an adjunct to the Clean Air Act’s primary goal of safeguarding 
public health, not an end in itself. A legitimate desire for accuracy and objectivity in repre¬ 
senting risk can induce such an obsession with numbers that too much energy is expended on 
representing the results of risk assessment in precise numerical form. Thus, new research 
might be commissioned because there is insufficient notice of how marginal the results would 
be in a given case or without consideration of new, less resource-intensive methods of 
providing essential inputs. 

Moreover, there might be a vast difference between having "the truth" and having enough 
information to enable a risk manager to choose the best course of action from the options 
available. The latter criterion is more applicable in a world with resource and time con¬ 
straints. Determining whether "enough information" exists to decide in turn implies the need 
to evaluate a full range of decisions. Thus, further improvement of a risk-assessment estimate 
might or might not be the most desirable course in a given situation, especially if the refine¬ 
ment is not likely to change the decision or if disproportionate resources have been directed to 
studying the risk at the expense of creating a full set of decision options from which to choose. 
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Comparisons of Risk 


It can be questioned whether risk assessment is sufficiently developed for the particular class 
of decisions regarding "offsets" or other tradable actions. In general, because of the substan¬ 
tial and varied degrees of model and parameter uncertainties in risk estimates, it is almost 
impossible to rank relative risks accurately unless the uncertainty in each risk is quantified or 
otherwise accounted for in the comparison. If the regulatory need for comparison of risks is 
imperative, one might attempt to compute the uncertainty distribution of the ratio of the two 
risks and choose from it one or more appropriate summary statistics. For example, one might 
determine in a given case that there is a 90% chance that chemical A is riskier than chemical B 
and a 50% chance that it is at least 10 times as risky. Also, if EPA decides to undertake the 
proposed iterative approach to risk assessment, it will not be possible to apply this kind of 
ratio comparison to estimates derived from different tiers of analysis. That is because the 
analyses at each level will be conducted differently and will produce risk estimates of differing 
accuracy and conservatism. The same might be true of aggregation of risks associated with 
different exposures. 

Even more difficult is the issue of the relative degrees of reliability in the risk figures being 
compared. Is it appropriate, for example, to compare actuarial risks with modeled risks? 

Those and other difficulties suggest that EPA should pay more attention than it now does to the 
appropriateness of various procedures for risk comparison. A scientifically sound way to do 
this would be to modify risk-assessment procedures to characterize more specifically the 
uncertainties in each comparison of risks—some larger, some smaller than the uncertainties in 
individual risk assessments—and this could be done across tiers. 


Risk Management and Research 

Improved cooperation between EPA's Office of Air Quality Planning and Standards 
(OAQPS), which conducts the regulatory work of the air program, and its Office of Research 
and Development (ORD), which conducts research and revises the risk-assessment guidelines, 
would be helpful in ensuring that research needs of the risk-management side were met by the 
research side. For example, the two groups might jointly publish a research agenda on 
hazardous air pollutants, submit the agenda for public comment and SAB review, publish a 
final agenda based on these comments, and then report annually on how much progress has 
been made on the agenda. EPA should have a review and research-management system that 
catalogs risk-assessment weaknesses as identified by the SAB and other peer-review activities 
and that helps to direct research within EPA (and to guide strategies in other federal and state 
agencies and in the private sector) to remedy the weaknesses when the importance of a risk 
assessment justifies the expenditure of research funds. In many cases, the regulated parties 
may be willing to fund research that will enable health-protective default options in risk 
assessment to be replaced by more complex and less conservative alternatives. EPA will need 
to maintain its own substantial research capability to understand and evaluate advances in risk 
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assessment. In some cases, EPA will want to support targeted risk-assessment research and 
data collection on specific chemicals that could lead to revisions in risk assessments of such 
chemicals. Situations might be discovered where current risk-assessment practice is underesti¬ 
mating health risk or where the information base for a chemical is not sufficient to allow 
regulation to proceed. 

Present EPA practice is to remove IRIS listings while cancer potencies or RfCs are under 
review. This practice is frustrating to non-EPA users, not only because the information 
becomes inaccessible, but also because EPA has been reluctant to state when such information 
will be returned to the system. The committee believes that a better practice would be for 
EPA to retain listings in the database, inform users that it is conducting a review, and perhaps 
include alternatives that can be used in the interim as the basis for calculated cancer potencies 
and RfCs. The narrative supporting the information on each chemical in IRIS should inform 
users about the assumptions underlying each calculation, about sources of data and judgments 
about uncertainty and variability, and about research under way to improve risk assessment on 
the chemical in support of future regulatory decisions. 


Risk Assessment as a Policy Guide 

Allocations of public-health resources reflect, among other things, some estimate of the 
potential benefits from health improvements achieved, and risk assessment is an important tool 
for understanding potential public-health impacts. Seen from this perspective, risk assessment 
should be a principal component of public-health and regulatory programs. Risk-management 
approaches will differ, perhaps greatly, depending on political choices. But establishing the 
relative impacts of various resource-allocation for achieving risk reduction, by continuing 
pursuit of comprehensive assessments of risks, should always be an objective. 

For example, the committee is concerned that neither Section 112 nor other legislation 
provides for appropriate control of toxic emissions from mobile and indoor sources. There is 
strong evidence that public exposure to chemicals (and radiation) in these settings can give rise 
to higher public-health risk in many cases than outdoor exposure due to stationary-source 
emissions. 

Focusing regulation on the source, rather than on the overall reduction of the pollutant (and 
its potential risk to public health), is unlikely to be very cost-effective in reducing disease, 
although it might effectively reduce high individual risks and reduce public concern over 
involuntary exposures. Given limited funds for both the analysis and control of environmental 
problems, some believe that EPA should focus on environmental toxicants that pose the 
greatest public-health threat. 


to 

o 

•fe 

CO 


Chapter 12: Implementation 


12-17 


Source: https://www.industrydocuments.ucsf.edu/docs/jnflOOOO 





Social and Cultural Factors 


Although the principle of maximal risk reduction is of central importance, some social and 
cultural factors that might introduce different risk-management priorities also need to be 
considered. 

First, it is apparent from many studies that people's perceptions of relative risk do not 
always match those of technical experts. When it comes to comparing risks, most people 
evaluate not only the mathematical probability that an adverse outcome will occur—the 
principal concern of the technical expert—but also other less tangible features of the risk 
context, most of which are not generally considered by the risk assessor. These other 
concerns should be expressed and reflected at the stage of risk management. 

For example, people generally feel greater anxiety about relatively low probability events 
with catastrophic outcomes (such as an airplane crash) than about higher-probability activities 
that take only a few lives at a time (such as an automobile collision). People are reluctant to 
accept risks, no matter how small, unless they feel that the risky activity or exposure provides 
some personal benefit. Risks believed to be imposed by others are less well tolerated than 
those voluntarily assumed. In a related vein, risks perceived as being of natural origin are less 
threatening than risks created by other human beings. Risks that scientists do not understand 
well, and over which they publicly disagree, are more feared than those about which scientific 
consensus is strong. Buttressing these observations is additional research that helps us to 
understand why people, and their governments, seem at times much more anxious about, and 
willing to act against, the risks associated with industrial chemicals than risks that scientists 
believe are more important from a public-health perspective (Slovic, 1987). We know, for 
example, that public perceptions of the need for regulation are influenced by such concerns as 
people's trust in government, their experience with experts' reassurances, and their views 
about social justice. When public opinion appears to be exaggerating the risks associated with 
industrial products, their fear might in fact be founded on an understandable mistrust of the 
institutional context in which those risks are produced, assessed, and eventually controlled. 


Summary 

Apart from its specific findings and recommendations, the committee's report is dominated 
by a number of central themes: 

1. EPA should retain its conservative, default-based approach to risk assessment for the 
purposes of screening analysis for standard-setting; however, a number of corrective actions 
are necessary for this approach to work properly. 

2. EPA should rely more on scientific judgment and less on rigid procedures by taking an 
iterative approach to its work. Such judgment demands more understanding of the relationship 
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between risk assessment and risk management and a creative but disciplined blending of the 
two. 

3. The iterative approach proposed by the committee provides the ability to make im¬ 
provements in both the models and data used in its analysis. However, in order for this 
approach to work properly, EPA needs to provide justification for its current defaults and set 
up a procedure such as that proposed in the report that permits departures from the default 
options. 

4. When reporting estimates of risk to decision-makers and the public, EPA should report 
not only point estimates of risk but also the sources and magnitudes of uncertainty associated 
with these estimates. 


Findings and Recommendations 

General findings and recommendations regarding implementation and risk management are 
presented below. 


Tiered vs. Iterative Risk Assessment 

EPA proposes to adopt a tiered risk-assessment approach that will begin with a "lookup" 
table and move to deeper analysis with the amount of conservatism generally decreasing as 
estimated uncertainty decreases. 

• Rather than a tiered risk-assessment process, EPA should develop the ability to conduct 
iterative risk assessments, allowing improvements in the process until the risk, assessed 
conservatively, is below the applicable decision-making level (e.g., 1 x 10* 6 , etc.); until 
further improvements would not significantly change the risk estimate; or until EPA, the 
source, or the public determines that the stakes are not high enough to warrant further 
analysis. 


Verification of Amount of Risk-Assessment Conservatism 

In its tiered approach, EPA plans to use exposure models developed and validated for 
criteria pollutants, but not fully evaluated for the broader group of situations including 
hazardous air pollutants. In particular, it has not shown that analysis conducted with a simple, 
single Gaussian-plume approach with the generic worst-case conditions will necessarily be 
conservative over all situations in which it would be applied. 
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• Until the accuracy and conservatism of the proposed models can be evaluated, EPA 
should consider beginning at Tier 3, where site-specific data will provide better estimates 
needed for such key decisions as delisting, priority-setting, and residual-risk decisions. 


Full Set of Exposure Models 

Even at Tier 3, EPA plans to use a Gaussian-plume model that does not hold over complex 
terrain. EPA's complex-terrain models focus on tall stacks, rather than the effects of hills or 
valleys, and emissions from a low point or area source disperse poorly in these models. 

• The committee recommends no specific model, but EPA should look beyond the set of 
models it now uses to find the best possible models of dispersion of hazardous air pollutants in 
complex terrain. 


IRIS Data Quality 

EPA plans to use IRIS as the database for as many as possible of the 189 Section 112 
chemicals. The IRIS database has quality problems and is not fully referenced. 

• EPA should enhance and expand the references in the data files on each chemical and 
include information on risk-assessment weaknesses for each chemical and the research needed 
to remedy such weaknesses. In addition, EPA should expand its efforts to ensure that IRIS 
maintains a high level of data quality. The chemical-specific files in IRIS should include 
references and brief summaries of EPA health-assessment documents and other major risk 
assessments of the chemicals carried out by the agency, reviews of these risk assessments by 
the EPA Science Advisory Board, and the agency's responses to the SAB reviews. Important 
risk assessments carried out by other government agencies or private parties should also be 
referenced and summarized. 


Toxicity Data Development 
Some of the 189 chemicals lack cancer potencies or RfCs. 

• If IRIS does not contain a cancer potency or RfC, EPA should develop a procedure for 
making crude screening estimates. These estimates should generally not be used for regulation, 
but only as a means of setting research priorities for carrying out the animal studies from 
which cancer potencies and RfCs could be calculated with EPA standard default methods. 

EPA should develop a summary of health-risk research needs from a review of the IRIS files 
on the 189 chemicals. EPA should determine which research is most important, how much of 
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it is likely to be carried out by other parties, and what research should be carried out by EPA 
and other federal agencies under their mandates to protect public health. 


Full Data Set for Priority-Setting 

EPA often appears to base priorities on the simple availability of data on a particular 
chemical. 

c At a minimum, EPA should compile for each of the 189 chemicals an inventory of the 
existing and relevant chemical, toxicologic, clinical, and epidemiologic literature. For each 
specific chemical, EPA should have at a minimum a structure-activity evaluation; and for each 
important mixture, it should complete an analysis of available short-term toxicity tests (such as 
the Ames test). If review of toxicity information suggests a possible need for regulation to 
protect human health, it should develop aggregate emission data and estimates of populations 
potentially exposed. 


Iterative Priortty-Setting 

EPA sometimes appears to base its priorities on a one-time analysis of incomplete and 
preliminary data. 

• EPA should take an iterative approach to gathering and evaluating existing evidence to 
use in a level of risk assessment for each of the 189 chemicals that is appropriate for the 
quality and quantity of available evidence and the most realistic scientific judgment of potential 
human health risks. On the basis of that evidence, EPA should further maintain a continuing 
oversight of new scientific results so that it can identify needs to re-examine chemicals that it 
has already assessed. 


Full and Complete Documentation of Priortty-Setting 

EPA does not always clearly communicate the methods and data on which it bases its 
priority-setting analysis. In addition, emission, exposure, and toxicity information is not often 
collected in the same database. 

4 

• Once EPA’s preliminary priority-setting analyses are completed for a chemical on the list, 
a description of the assessment process used, the findings, and the emission, exposure, and 
toxicity information should be placed in one location in the public domain (e.g., in IRIS). 
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Guidelines vs. Requirements 

EPA and others often interpret the term risk assessment as a specific methodologic approach 
to extrapolating from sets of human and animal carcinogenicity data, often obtained in intense 
exposures, to quantitative estimates of carcinogenic risk associated with the (typically) much 
lower exposures experienced by human populations. 

• EPA should recognize that the conduct of risk assessment does not require any specific 
methodologic approach and that it is best seen not as a number or even a document, but as a 
way to organize knowledge regarding potentially hazardous activities or substances and to 
facilitate the systematic analysis of the risks that those activities or substances might pose 
under specified conditions. The limitations of risk assessment thus broadly conceived will be 
clearly seen as resulting from limitations in our current state of scientific understanding. 
Therefore, risk-assessment guidelines should be just that—guidelines, not requirements. EPA 
should give specific long-term attention to ways to improve this process, including changes in 
guidelines. 


Process for Public Review and Comment 

EPA does not always provide a method by which industry, environmental groups, or the 
general public can raise questions regarding the scientific basis of a decision made by EPA 
during the risk-assessment process. 

• EPA should provide a process for public review and comment with a requirement that it 
respond, so that outside parties can be assured that the methods used in risk assessments are 
scientifically justifiable. 


Petitions for Departure from Default Options 

EPA does not have a procedural mechanism that allows those outside EPA to petition for 
departures from default options. 

• EPA should develop a formal process to allow those outside the agency to petition for 
departures. 


Iterative Uncertainty Analysis 

Because EPA often fails to characterize fully the uncertainty in risk assessments, inappropri¬ 
ate decisions and insufficiently or excessively conservative analyses can result. 
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• The committee believes that the uncertainty in a risk estimate can be handled through an 
iterative process with the following parts: conduct a conservative screening analysis, conduct a 
default-uncertainty analysis, and conduct testing or analysis to develop site-specific probability 
distributions for each important input. The key factor in deciding to increase the intensiveness 
of uncertainty analysis should be the extent to which changes in estimates of costs and risks 
could affect risk-management decisions. 


Risk Assessment vs. Risk Management 

The principle of separation of risk assessment from risk management has led to systematic 
downplaying of the science-policy judgments embedded in risk assessment. Risk assessment 
accordingly is sometimes mistakenly perceived as a search for "truth" independent of 
management concerns. 

• EPA should increase institutional and intellectual linkages between risk assessment and 
risk management so as to create better harmony between the science-policy components of risk 
assessment and the broader policy objectives of risk management. This must be done in a 
way that fully protects the accuracy, objectivity, and integrity of its risk assessments—but the 
committee does not see these two aims as incompatible. Interagency and public understanding 
would be served by the preparation and release of a report on the science-policy issues and 
decisions that affect EPA's risk-assessment and risk-management practices. 


Comparisons of Risk 

EPA often does not elucidate all relevant considerations of technical accuracy when it 
compares and ranks risks. 

• EPA should further develop its methods for risk comparison, taking account of such 
factors as differing degrees of uncertainty and of conservatism in different categories of risk 
assessment. 


Policy Focus on Stationary Sources 

Title HI focuses primarily on outdoor stationary sources of hazardous air pollutants and does 
not consider to indoor or mobile sources of those pollutants. 

• EPA should clearly communicate to Congress that emissions and exposure, and thus the 
aggregate risk to the public, related to indoor and mobile sources might well be higher than 
those related to stationary sources. 
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Risk Management and Research 


EPA does not appear to use risk assessment adequately as a guide to research and might 
abandon some important risk-assessment and regulatory efforts prematurely because of data 
inadequacies. 

• The conduct of risk assessment reveals major scientific uncertainties in a highly system¬ 
atic way, so it is an excellent guide to the development of research programs to improve 
knowledge of risk. EPA should, therefore, not abandon risk assessments when data are 
inadequate, but should seek to explore the implications for research. Risk-assessment uncer¬ 
tainties can also help to determine the urgency with which such research should be developed. 
In particular, improved cooperation between EPA's Office of Air Quality Planning and 
Standards (OAQPS) and its Office of Research and Development (ORD) through such actions 
as joint publication of a research agenda on hazardous air pollutants would be most helpful. 
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